Semi-Supervised Keyword Spotting in Arabic Speech Using Self-Training Ensembles

Arabic speech recognition suffers from the scarcity of properly labeled data. In this project, we introduce a pipeline that performs semi-supervised segmentation of audio then— after hand-labeling a small dataset—feeds labeled segments to a supervised learning framework to select, through many rounds of hyperparameter optimization, an ensemble of models to infer labels for a larger dataset; using which we improved the keyword spotter’s F1 score from 75.85% (using a baseline model) to 90.91% on a ground-truth test set. We picked the keyword na‘am (yes) to spot; we defined the system’s input as an audio file of an utterance and the output as a binary label: keyword or filler. Keywords—Arabic; Acoustic Model; Ensemble Learning; Extra Trees; Gradient Boosting; K-Nearest Neighbors; Keyword Spotting; King Saud University Arabic Speech Database; Random Forests; Segmentation; Self-Training; Semi-Supervised Learning; Speech Recognition; Support Vector Machines; West Point Arabic Speech